Focus adjustment method for a virtual reality headset

ABSTRACT

A virtual reality headset displays a three-dimensional (3D) virtual scene and includes a varifocal element to dynamically adjust a focal length of an optics block included in the virtual reality headset based on a location in the virtual scene where the user is looking. The headset tracks a user&#39;s eyes to approximate gaze lines and determines a plane of focus for a frame of the virtual scene as the intersection of the gaze lines. The varifocal element adjusts the focal length of the optics block so the optics block is focused at the plane of focus, which keeps the user&#39;s eyes in a zone of comfort as vergence and accommodation change. Based on the plane of focus, the virtual reality headset may provide depth cues, such as depth of field blur, to planes in the virtual scene deeper in the user&#39;s field of view than the plane of focus.

BACKGROUND

The present disclosure generally relates to enhancing images fromelectronic displays, and specifically to varying the focal length ofoptics to enhance the images.

Virtual reality (VR) headset can be used to simulate virtualenvironments. For example, stereoscopic images can be displayed on anelectronic display inside the headset to simulate the illusion of depthand head tracking sensors can be used to estimate what portion of thevirtual environment is being viewed by the user. Such a simulation,however, can cause visual fatigue and nausea resulting from an inabilityof existing headsets to correctly render or otherwise compensate forvergence and accommodation conflicts.

SUMMARY

A virtual reality headset automatically adjusts its focus based on alocation within a virtual scene presented by the virtual reality headsetthat the user views. A three-dimensional (3D) virtual scene is presentedon an electronic display element (e.g., a screen) of the virtual realityheadset and a focal length of an optics block that directs image lightfrom the electronic display element towards eyes of the user is adjustedusing a varifocal element (e.g., an element that mechanically changes adistance between a lens system in the optics block and the electronicdisplay element, an element that changes shape of one or more lenses inthe lens system in the optics block, etc.) based on a location or objectwithin the virtual scene where the user is looking. For example, thevirtual reality headset tracks a user's eyes to approximate gaze linesand determines a gaze point including a vergence depth as an estimatedpoint of intersection of the gaze lines. The gaze point identifying anobject or plane of focus for a particular frame of the virtual scenepresented to the user by the virtual reality headset.

The gaze lines often do not intersect and are an approximation based onthe location or position of the user's eyes, the vergence depth of thegaze point estimated from the gaze lines may be further refined orfiltered using geometric data for the virtual scene presented by thevirtual reality headset. Based on the positions of the user's head andeyes, the virtual reality headset identifies a location or an objectwithin the virtual scene presented by the virtual headset where the useris looking. Based on the location or the object within the virtual scenewhere the user is looking and geometric data for the virtual scene(i.e., virtual distances between objects in the virtual scene), thevirtual reality headset corrects the estimated vergence depth and gazepoint.

In various embodiments, the varifocal element adjusts the focal lengthof the optics block to focus the optics block at the corrected estimatedvergence depth for the gaze point to keep the user's eyes in a zone ofcomfort as vergence and accommodation change. Further, given the objector plane of focus at the corrected estimated vergence depth, the virtualreality headset may dynamically add depth of field blur based on theobject or the plane of focus when rendering the virtual scene to providea depth cue.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example virtual reality system, in accordance with atleast one embodiment.

FIG. 2 shows a diagram of a virtual reality headset, in accordance withat least one embodiment.

FIG. 3 shows a virtual reality headset, in accordance with at least oneembodiment.

FIG. 4 shows an example process for mitigating vergence-accommodationconflict by adjusting the focal length of an optics block of a virtualreality headset, in accordance with at least one embodiment.

FIG. 5 shows a cross section of a virtual reality headset including acamera for tracking eye position, in accordance with at least oneembodiment.

FIG. 6 shows an example process for filtering a vergence depth based onscene geometry, in accordance with at least one embodiment.

FIG. 7A shows the relationship between vergence and eye focal length inthe real world.

FIG. 7B shows the conflict between vergence and eye focal length in athree-dimensional display.

FIGS. 8A and 8B show an example process for adjusting the focal lengthof an optics block of a virtual reality headset by varying the distancebetween a display screen and the optics block using a varifocal element,in accordance with at least one embodiment.

FIGS. 9A and 9B show an example process for adjusting the focal lengthby changing the shape or optical path length of the optics block of avirtual reality headset using a varifocal element, in accordance with atleast one embodiment.

The figures depict embodiments of the present disclosure for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles, or benefits touted, of the disclosure described herein.

DETAILED DESCRIPTION System Overview

FIG. 1 is virtual reality (VR) system environment in which a VR console150 operates. In this example, the VR system environment includes VRheadset 100, imaging device 160, and VR input interface 170, which areeach coupled to VR console 150. While FIG. 1 shows a single VR headset100, a single imaging device 160, and a single VR input interface 170,in other embodiments, any number of these components may be included inthe system. For example, there may be multiple VR headsets 100 eachhaving an associated VR input interface 170 and being monitored by oneor more imaging devices 160, with each VR headset 100, VR inputinterface 170, and imaging devices 160 communicating with the VR console150. In alternative configurations, different and/or additionalcomponents may also be included in the VR system environment.

VR headset 100 is a Head-Mounted Display (HIVID) that presents contentto a user. Example content includes images, video, audio, or somecombination thereof. Audio content may be presented via a separatedevice (e.g., speakers and/or headphones) external to VR headset 100that receives audio information from VR headset 100, VR console 150, orboth. VR headset 100 includes electronic display 102, optics block 104,varifocal actuation block 106, focus prediction module 108, eye trackingmodule 110, vergence processing module 112, one or more locators 114,internal measurement unit (IMU) 116, head tracking sensors 118, andscene rendering module 120.

Optics block 104 directs light from electronic display 102 to an exitpupil for viewing by a user using one or more optical elements, such asapertures, Fresnel lenses, convex lenses, concave lenses, filters, andso forth, and may include combinations of different optical elements. Insome embodiments, one or more optical elements in optics block 104 mayhave one or more coatings, such as anti-reflective coatings.Magnification of the image light by optics block 104 allows electronicdisplay 102 to be physically smaller, weigh less, and consume less powerthan larger displays. Additionally, magnification of the image light mayincrease a field of view of the displayed content. For example, thefield of view of the displayed content is such that the displayedcontent is presented using almost all (e.g., 150 degrees diagonal), andin some cases all, of the user's field of view.

Optics block 104 may be designed to correct one or more optical errors.Examples of optical errors include: barrel distortion, pincushiondistortion, longitudinal chromatic aberration, transverse chromaticaberration, spherical aberration, comatic aberration, field curvature,astigmatism, and so forth. In some embodiments, content provided toelectronic display 102 for display is pre-distorted, and optics block104 corrects the distortion when it receives image light from electronicdisplay 102 generated based on the content.

Varifocal actuation block 106 includes a varifocal element that causesoptics block 104 to vary the focal length (or optical power) of VRheadset 100 keep a user's eyes in a zone of comfort as vergence andaccommodation change. In one embodiment, varifocal actuation block 106physically changes the distance between electronic display 102 andoptical block 104 by moving electronic display 102 or optical block 104(or both). Alternatively, varifocal actuation block 106 changes thefocal length of optics block 104 by adjusting one or more properties ofone or more lenses. Example properties of a lens adjusted by thevarifocal actuation block include: an optical path length, an index ofrefraction of a lens medium, a shape of a lens, and so forth. Forexample, varifocal actuation block 106 changes the focal length of theone or more lenses using shape-changing polymer lenses, electrowettingmethods with liquid lenses, Alvarez-Lohmann lenses, deformable membranemirrors, liquid crystal (electroactive) lenses, or phase-only spatiallight modulators (SLMs), or any other suitable component. Additionally,moving or translating two lenses relative to each other may also be usedto change the focal length of VR headset 100. Thus, varifocal actuationblock 106 may include actuators or motors that move electronic display102 and/or optical block 104 on a track to change the distance betweenthem or may include actuators and other components or mechanisms forchanging the properties of one or more lenses included in optics block104. Varifocal actuation block 106 may be separate from or integratedinto optics block 104 in various embodiments.

Each state of optics block 104 corresponds to a focal length of VRheadset 110 or to a combination of the focal length and eye positionrelative to optics block 104 (as discussed further below). In operation,optics block 104 may move in a range of ˜5mm with a positional accuracyof ˜5 μm for a granularity of around 1000 focal lengths, correspondingto 1000 states of optics block 104. Any number of states could beprovided; however, a limited number of states accommodate thesensitivity of the human eye, allowing some embodiments to include fewerfocal lengths. For example, a first state corresponds to a focal lengthof a theoretical infinity meters (0 diopter), a second state correspondsto a focal length of 2.0 meters (0.5 diopter), a third state correspondsto a focal length of 1.0 meters (1 diopter), a fourth state correspondsto a focal length of 0.5 meters (1 diopter), a fifth state correspondsto a focal length of 0.333 meters (3 diopter), and a sixth statecorresponds to a focal length of 0.250 meters (4 diopter). Varifocalactuation block 106, thus, sets and changes the state of optics block104 to achieve a desired focal length.

Focus prediction module 108 is an encoder including logic that tracksthe state of optics block 104 to predict to one or more future states orlocations of optics block 104. For example, focus prediction module 108accumulates historical information corresponding to previous states ofoptics block 104 and predicts a future state of optics block 104 basedon the previous states. Because rendering of a virtual scene by VRheadset 100 is adjusted based on the state of optics block 104, thepredicted state allows scene rendering module 120, further describedbelow, to determine an adjustment to apply to the virtual scene for aparticular frame. Accordingly, focus prediction module 108 communicatesinformation describing a predicted state of optics block 104 for a frameto scene rendering module 120. Adjustments for the different states ofoptics block 104 performed by scene rendering module 120 are furtherdescribed below.

Eye tracking module 110 tracks an eye position and eye movement of auser of VR headset 100. A camera or other optical sensor inside VRheadset 100 captures image information of a user's eyes, and eyetracking module 110 uses the captured information to determineinterpupillary distance, interocular distance, a three-dimensional (3D)position of each eye relative to VR headset 100 (e.g., for distortionadjustment purposes), including a magnitude of torsion and rotation(i.e., roll, pitch, and yaw) and gaze directions for each eye. In oneexample, infrared light is emitted within VR headset 100 and reflectedfrom each eye. The reflected light is received or detected by the cameraand analyzed to extract eye rotation from changes in the infrared lightreflected by each eye. Many methods for tracking the eyes of a user canbe used by eye tracking module 110. Accordingly, eye tracking module 110may track up to six degrees of freedom of each eye (i.e., 3D position,roll, pitch, and yaw) and at least a subset of the tracked quantitiesmay be combined from two eyes of a user to estimate a gaze point (i.e.,a 3D location or position in the virtual scene where the user islooking). For example, eye tracking module 110 integrates informationfrom past measurements, measurements identifying a position of a user'shead, and 3D information describing a scene presented by electronicdisplay element 102. Thus, information for the position and orientationof the user's eyes is used to determine the gaze point in a virtualscene presented by VR headset 100 where the user is looking.

Further, distance between a pupil and optics block 104 changes as theeye moves to look in different directions. The varying distance betweenpupil and optics block 104 as viewing direction changes is referred toas “pupil swim” and contributes to distortion perceived by the user as aresult of light focusing in different locations as the distance betweenpupil and optics block 104. Accordingly, measuring distortion adifferent eye positions and pupil distances relative to optics block 104and generating distortion corrections for different positions anddistances allows mitigation of distortion caused by “pupil swim” bytracking the 3D position of a user's eyes and applying a distortioncorrection corresponding to the 3D position of each of the user's eye ata given point in time. Thus, knowing the 3D position of each of a user'seyes allows for the mitigation of distortion caused by changes in thedistance between the pupil of the eye and optics block 104 by applying adistortion correction for each 3D eye position.

Vergence processing module 112 determines a vergence depth of a user'sgaze based on the gaze point or an estimated intersection of the gazelines determined by eye tracking module 110. Vergence is thesimultaneous movement or rotation of both eyes in opposite directions tomaintain single binocular vision, which is naturally and automaticallyperformed by the human eye. Thus, a location where a user's eyes areverged is where the user is looking and is also typically the locationwhere the user's eyes are focused. For example, vergence processingmodule 112 triangulates the gaze lines to estimate a distance or depthfrom the user associated with intersection of the gaze lines. The depthassociated with intersection of the gaze lines can then be used as anapproximation for the accommodation distance, which identifies adistance from the user where the user's eyes are directed. Thus, thevergence distance allows determination of a location where the user'seyes should be focused and a depth from the user's eyes at which theeyes are focused, thereby, providing information, such as an object orplane of focus, for rendering adjustments to the virtual scene.

In some embodiments, rather than provide accommodation for the eye at adetermined vergence depth, accommodation may be directly determined by awavefront sensor, such as a Shack-Hartmann wavefront sensor; hence, astate of optics block 104 may be a function of the vergence oraccommodation depth and the 3D position of each eye, so optics block 104brings objects in a scene presented by electronic display element 102into focus for a user viewing the scene. Further, vergence andaccommodation information may be combined to focus optics block 104 andto render synthetic depth of field blur.

Locators 114 are objects located in specific positions on VR headset 100relative to one another and relative to a specific reference point on VRheadset 100. Locator 114 may be a light emitting diode (LED), a cornercube reflector, a reflective marker, a type of light source thatcontrasts with an environment in which VR headset 100 operates, or somecombination thereof. Active locators 114 (i.e., an LED or other type oflight emitting device) may emit light in the visible band (˜380 nm to750 nm), in the infrared (IR) band (˜750 nm to 1 mm), in the ultravioletband (10 nm to 380 nm), some other portion of the electromagneticspectrum, or some combination thereof.

Locators 114 can be located beneath an outer surface of VR headset 100,which is transparent to the wavelengths of light emitted or reflected bylocators 114 or is thin enough not to substantially attenuate thewavelengths of light emitted or reflected by locators 114. Further, theouter surface or other portions of VR headset 100 can be opaque in thevisible band of wavelengths of light. Thus, locators 114 may emit lightin the IR band while under an outer surface of VR headset 100 that istransparent in the IR band but opaque in the visible band.

IMU 116 is an electronic device that generates fast calibration databased on measurement signals received from one or more of head trackingsensors 118, which generate one or more measurement signals in responseto motion of VR headset 100. Examples of head tracking sensors 118include accelerometers, gyroscopes, magnetometers, other sensorssuitable for detecting motion, correcting error associated with IMU 116,or some combination thereof. Head tracking sensors 118 may be locatedexternal to IMU 116, internal to IMU 116, or some combination thereof.

Based on the measurement signals from head tracking sensors 118, IMU 116generates fast calibration data indicating an estimated position of VRheadset 100 relative to an initial position of VR headset 100. Forexample, head tracking sensors 118 include multiple accelerometers tomeasure translational motion (forward/back, up/down, left/right) andmultiple gyroscopes to measure rotational motion (e.g., pitch, yaw, androll). IMU 116 can, for example, rapidly sample the measurement signalsand calculate the estimated position of VR headset 100 from the sampleddata. For example, IMU 116 integrates measurement signals received fromthe accelerometers over time to estimate a velocity vector andintegrates the velocity vector over time to determine an estimatedposition of a reference point on VR headset 100. The reference point isa point that may be used to describe the position of VR headset 100.While the reference point may generally be defined as a point in space,in various embodiments, reference point is defined as a point within VRheadset 100 (e.g., a center of the IMU 130). Alternatively, IMU 116provides the sampled measurement signals to VR console 150, whichdetermines the fast calibration data.

IMU 116 can additionally receive one or more calibration parameters fromVR console 150. As further discussed below, the one or more calibrationparameters are used to maintain tracking of VR headset 100. Based on areceived calibration parameter, IMU 116 may adjust one or more IMUparameters (e.g., sample rate). In some embodiments, certain calibrationparameters cause IMU 116 to update an initial position of the referencepoint to correspond to a next calibrated position of the referencepoint. Updating the initial position of the reference point as the nextcalibrated position of the reference point helps reduce accumulatederror associated with determining the estimated position. Theaccumulated error, also referred to as drift error, causes the estimatedposition of the reference point to “drift” away from the actual positionof the reference point over time.

Scene render module 120 receives content for the virtual scene from VRengine 156 and provides the content for display on electronic display102. Additionally, scene render module 120 can adjust the content basedon information from focus prediction module 108, vergence processingmodule 112, IMU 116, and head tracking sensors 118. For example, uponreceiving the content from VR engine 156, scene render module 120adjusts the content based on the predicted state (i.e., eye position andfocal length) of optics block 104 received from focus prediction module108 by adding a correction or pre-distortion into rendering of thevirtual scene to compensate or correct for the distortion caused by thepredicted state of optics block 104. Scene render module 120 may alsoadd depth of field blur based on the user's gaze, vergence depth (oraccommodation depth) received from vergence processing module 112, ormeasured properties of the user's eye (e.g., 3D position of the eye,etc.). Additionally, scene render module 120 determines a portion of thecontent to be displayed on electronic display 102 based on one or moreof tracking module 154, head tracking sensors 118, or IMU 116, asdescribed further below.

Imaging device 160 generates slow calibration data in accordance withcalibration parameters received from VR console 150. Slow calibrationdata includes one or more images showing observed positions of locators114 that are detectable by imaging device 160. Imaging device 160 mayinclude one or more cameras, one or more video cameras, other devicescapable of capturing images including one or more locators 114, or somecombination thereof. Additionally, imaging device 160 may include one ormore filters (e.g., for increasing signal to noise ratio). Imagingdevice 160 is configured to detect light emitted or reflected fromlocators 114 in a field of view of imaging device 160. In embodimentswhere locators 114 include passive elements (e.g., a retroreflector),imaging device 160 may include a light source that illuminates some orall of locators 114, which retro-reflect the light towards the lightsource in imaging device 160. Slow calibration data is communicated fromimaging device 160 to VR console 150, and imaging device 160 receivesone or more calibration parameters from VR console 150 to adjust one ormore imaging parameters (e.g., focal length, focus, frame rate, ISO,sensor temperature, shutter speed, aperture, etc.).

VR input interface 170 is a device that allows a user to send actionrequests to VR console 150. An action request is a request to perform aparticular action. For example, an action request may be to start or endan application or to perform a particular action within the application.VR input interface 170 may include one or more input devices. Exampleinput devices include a keyboard, a mouse, a game controller, or anyother suitable device for receiving action requests and communicatingthe received action requests to VR console 150. An action requestreceived by VR input interface 170 is communicated to VR console 150,which performs an action corresponding to the action request. In someembodiments, VR input interface 170 may provide haptic feedback to theuser in accordance with instructions received from VR console 150. Forexample, haptic feedback is provided by the VR input interface 170 whenan action request is received, or VR console 150 communicatesinstructions to VR input interface 170 causing VR input interface 170 togenerate haptic feedback when VR console 150 performs an action.

VR console 150 provides content to VR headset 100 for presentation tothe user in accordance with information received from imaging device160, VR headset 100, or VR input interface 170. In the example shown inFIG. 1, VR console 150 includes application store 152, tracking module154, and virtual reality (VR) engine 156. Some embodiments of VR console150 have different or additional modules than those described inconjunction with FIG. 1. Similarly, the functions further describedbelow may be distributed among components of VR console 150 in adifferent manner than is described here.

Application store 152 stores one or more applications for execution byVR console 150. An application is a group of instructions, that whenexecuted by a processor, generates content for presentation to the user.Content generated by an application may be in response to inputsreceived from the user via movement of VR headset 100 or VR interfacedevice 170. Examples of applications include gaming applications,conferencing applications, video playback application, or other suitableapplications.

Tracking module 154 calibrates the VR system using one or morecalibration parameters and may adjust one or more calibration parametersto reduce error in determining position of VR headset 100. For example,tracking module 154 adjusts the focus of imaging device 160 to obtain amore accurate position for observed locators 114 on VR headset 100.Moreover, calibration performed by tracking module 154 also accounts forinformation received from IMU 116. Additionally, if tracking of VRheadset 100 is lost (e.g., imaging device 160 loses line of sight of atleast a threshold number of locators 114), tracking module 154re-calibrates some or all of the VR system components.

Additionally, tracking module 154 tracks the movement of VR headset 100using slow calibration information from imaging device 160 anddetermines positions of a reference point on VR headset 100 usingobserved locators from the slow calibration information and a model ofVR headset 100. Tracking module 154 also determines positions of thereference point on VR headset 100 using position information from thefast calibration information from IMU 116 on VR headset 100.Additionally, tracking module 154 may use portions of the fastcalibration information, the slow calibration information, or somecombination thereof, to predict a future location of VR headset 100,which is provided to VR engine 156.

VR engine 156 executes applications within the VR system and receivesposition information, acceleration information, velocity information,predicted future positions, or some combination thereof for VR headset100 from tracking module 154. Based on the received information, VRengine 156 determines content to provide to VR headset 100 forpresentation to the user, such as a virtual scene. For example, if thereceived information indicates that the user has looked to the left, VRengine 156 generates content for VR headset 100 that mirrors or tracksthe user's movement in a virtual environment. Additionally, VR engine156 performs an action within an application executing on VR console 150in response to an action request received from the VR input interface170 and provides feedback to the user that the action was performed. Theprovided feedback may be visual or audible feedback via VR headset 100or haptic feedback via VR input interface 170.

FIG. 2 is a diagram of VR headset 100, in accordance with at least oneembodiment. In this example, VR headset 100 includes a front rigid bodyand a band that goes around a user's head. The front rigid body includesone or more electronic display elements corresponding to electronicdisplay 102, IMU 116, head tracking sensors 118, and locators 114. Inthis example, head tracking sensors 118 are located within IMU 116.

Locators 114 are located in fixed positions on the front rigid bodyrelative to one another and relative to reference point 200. In thisexample, reference point 200 is located at the center of IMU 116. Eachof locators 114 emits light that is detectable by imaging device 160.Locators 114, or portions of locators 114, are located on a front side,a top side, a bottom side, a right side, and a left side of the frontrigid body, as shown FIG. 2. FIG. 3 shows a rear perspective viewlooking into VR headset 300 where an outer lens of optics block 104 isvisible. In this example, locators 114 are visible and provided, asdiscussed above, on the surface of VR headset 300 for detection byimaging device 160.

Focus Adjustment Method

FIG. 4 shows an embodiment of a process 400 for mitigatingvergence-accommodation conflict by adjusting the focal length of opticsblock 104 of virtual reality (VR) headset 100. As discussed above, avarifocal system may dynamically vary its focus to bring imagespresented to a user wearing VR headset 100 into focus, which keeps theuser's eyes in a zone of comfort as vergence and accommodation change.Additionally, eye tracking in combination with the variable focus of thevarifocal system allows blurring to be introduced as depth cues inimages presented by VR headset 100.

Accordingly, in the embodiment shown by FIG. 4, a position, anorientation, and/or a movement of VR headset 100 are determined 402 by acombination of locators 114, IMU 116, head tracking sensors 118,imagining device 160, and tracking module 154, as described above inconjunction with FIG. 1. Portions of a virtual scene presented by VRheadset 100 are mapped to various positions and orientations of VRheadset 100. Thus, a portion of the virtual scene currently viewed by auser is determined 404 based on the position, orientation, and movementof VR headset 100. After determining 404 the portion of the virtualscene being viewed by the user, the VR system may then determine alocation or an object within the determined portion at which the user islooking to adjust focus for that location or object accordingly.

To determine the location or object within the determined portion of thevirtual scene at which the user is looking, VR headset 100 tracks theposition and location of the user's eyes. Thus, VR headset 100determines 406 an eye position for each eye of the user. For example, VRheadset 100 tracks at least a subset of the 3D position, roll, pitch,and yaw of each eye and uses these quantities to estimate a 3D gazepoint of each eye. Further, information from past eye positions,information describing a position of the user's head, and informationdescribing a scene presented to the user may also be used to estimatethe 3D gaze point of an eye in various embodiments. For example, FIG. 5shows a cross section of an embodiment of VR headset 100 that includescamera 502 for tracking the position of each eye 500. In this example,camera 502 captures images of the user's eyes and eye tracking module110 determines an output for each eye 500 and gaze lines 504corresponding to the gaze point or location where the user is lookingbased on the captured images.

Referring to FIGS. 4 and 5, vergence depth (d_(v)) 508 of the gaze pointfor the user is determined 410 based on an estimated intersection ofgaze lines 504. As shown in FIG. 5, gaze lines 504 converge or intersectat d_(v) 508, where object 506 is located. Because virtual distanceswithin the virtual scene are known to the VR system, the vergence depth508 can be filtered or verified to determine a more accurate vergencedepth for the virtual scene. For example, vergence depth 508 is anapproximation of the intersection of gaze lines 504, which arethemselves an approximation based on the position of a user's eyes 500.Gaze lines 504 do not always appear to accurately intersect. Thus,virtual distances within the virtual scene are compared 412 to thevergence depth for the portion of the virtual scene to generate afiltered vergence depth.

FIG. 6 shows an example process for filtering a vergence depth or gazepoint location based on scene geometry. In the example of FIG. 6, object606 is shown on a floor of virtual scene 600 and is the only object invirtual scene 600. Additionally, the geometry for virtual scene 600 isknown. For example, to provide a virtual environment enabling a user tointeract with that environment as if the user were actually in thevirtual environment, the dimensions of the environment corresponding toits geometry are accurately known to VR system. Thus, for a particularframe of virtual scene 600, the distance between the user and the wall,the user and object 606, and object 606 and the wall are known values.These known values allow the accuracy of the determined vergence depth.

In the example of FIG. 6, gaze lines 602 are initially verged atvergence depth d₁ (line 604), which corresponds to the estimatedvergence depth determined by vergence processing module 112. Vergenceprocessing module 112 receives data for virtual scene 600 that includingscene geometry data describing distances between objects in virtualscene 600 from scene render module 120 to verify the accuracy of theestimated vergence depth. The scene geometry data, in this example,indicates that virtual scene 600 includes object 606 at distance d₂(line 608) from the user. Vergence processing module 112 comparesdistance d₁ to d₂ to determine if they are equal. In the example of FIG.6, the distances d₁ and d₂ are unequal, but the difference is less thana threshold distance, indicating the estimated vergence depth (d₁) wasslightly inaccurate and that the vergence depth is more accurately d₂.Because, vergence processing module 112 obtains information indicatingthere are no other objects in virtual scene 600, vergence processingmodule 112 and filters or adjusts the estimated vergence depth d₁ tofiltered vergence depth d₂.

Determining a more accurate vergence depth or gaze point enables thevirtual scene to more accurately determine a user's object or plane offocus, allowing scene rendering module 120 to add depth of field blur toproper depths and/or objects in the virtual scene or otherwise modify tovirtual scene to appear more realistic. Further, if virtual scene 600 inFIG. 6 included multiple objects, the vergence processing module 112compares the estimated vergence depth to distances associated with atleast a subset of the objects. In one example, the minimum differencebetween distance to an object and the estimated vergence depth isdetermined to be the filtered vergence depth; however, other methods ofidentifying an object that specifies the filtered vergence depth may beused in various embodiments.

Returning to FIG. 4, a state of optics block 104 is determined 414 for aframe of the virtual scene based on states of optics block 140 duringpresentation of previous frames of the virtual scene. For example, focusprediction module 108 tracks the state of optics block 104 for variousframes of the virtual scene to predict to future a state of optics block104 for subsequent frames of the virtual scene. The predicted state ofoptics block 104 (e.g., a predicted location of optics block 104) allowsthe scene rendering module 114 to determine an adjustment to apply to aframe of the virtual scene so distortion caused by the predicted stateof optics block 104 corrects or cancels the applied adjustment ratherthan distorting the frame. Thus, based on the state of optics block 104,a distortion correction is determined 416 for application to a frame ofthe virtual scene to correct optical error introduced by the state ofoptics block 104.

FIG. 7A shows an example of how the human eye experiences vergence andaccommodation in the real world. Vergence is the simultaneous movementor rotation of both eyes in opposite directions to obtain or maintainsingle binocular vision and is connected to accommodation of the eye.Under normal conditions, changing the focus of the eyes to look at anobject at a different distance automatically causes vergence andaccommodation. In the example of FIG. 7A, the user is looking at realobject 700A (i.e., the user's eyes are verged on real object 700A andgaze lines from the user's eyes intersect at real object 700A.). As realobject 700A is moved closer to the user, as indicated by the arrow inFIG. 7A, each eye 702 rotates inward to stay verged on real object 700A.As real object 700A gets closer, eye 702 must “accommodate” for thecloser distance by reducing the power or focal length of eye 702 bychanging its shape. Thus, under normal conditions in the real world, thevergence depth (d_(v)) equals the focal length (d_(f)).

However, FIG. 7B shows an example conflict between vergence andaccommodation that can occur with some three-dimensional displays. Inthis example, a user is looking at virtual object 700B displayed on 3Delectronic screen 704; however, the user's eyes are verged on and gazelines from the user's eyes intersect at virtual object 700B, which is agreater distance from the user's eyes than 3D electronic screen 704. Asvirtual object 700B is rendered on 3D electronic display 704 to appearcloser to the user, each eye 702 again rotates inward to stay verged onvirtual object 700B, but the power or focal length of each eye is notreduced; hence, the user's eyes do not accommodate as in FIG. 7A. Thus,instead of reducing power or focal length to accommodate for the closervergence depth, eye 702 maintains accommodation at a distance associatedwith 3D electronic display 704. Thus, the vergence depth (d_(v)) oftendoes not equal the focal length (d_(f)) for the human eye for objectsdisplayed on 3D electronic displays. This discrepancy between vergencedepth and focal length is referred to as “vergence-accommodationconflict.” A user experiencing only vergence or accommodation and notboth will eventually experience some degree of fatigue and nausea, whichis undesirable desirable for virtual reality system creators. Changes invergence for a 3D electronic screen may be accommodated by a VR headsetdynamically adjusting the power of an optics block based on the vergencedepth (or predicted vergence depth).

Accordingly, referring again to FIG. 4, the focal length (or power) ofoptics block 104 is adjusted 418 for the presented frame of the virtualscene to provide accommodation for the generated filtered vergencedepth. FIGS. 8A and 8B show an example process for adjusting the focallength of optics block 104 by varying the distance between electronicdisplay 102 and optics block 104 using varifocal element 802. In theexample of FIGS. 8A and 8B, varifocal actuation block 106 includesvarifocal element 802, such as an actuator or motor and track 804, butmay also include other components enabling optics block 104, electronicdisplay 102, or both to move along track 804 to dynamically adjust theoptical power of optics block 104.

FIG. 8A shows an example of VR headset 100 providing s focus adjustmentfor frame n of a virtual scene. In this example, virtual scene includesobject 806 displayed on electronic display 102 at which the gaze of user800 is directed (i.e., verged). A virtual image of object 806 is locateda virtual distance d_(i), behind electronic display 102, from exit pupil810. In the example of FIG. 8A, optics block 104 is in position p_(i),which provides accommodation for distance d_(i) to enable comfortableviewing of object 806.

FIG. 8B shows VR headset 100 providing s focus adjustment for asubsequent frame n+1 of the virtual scene. In this example, user 800 mayhave repositioned its eyes to look at object 808 or object 808 quicklymoved toward user 800 in the virtual scene. As a result, the virtualimage of object 808 is located close to electronic display 102. Inresponse to the location of object 808 close to the electronic display102, which is closer than object 806 in FIG. 8A), eyes of user 800rotate inward to verge on object 808, causing vergence processing module112 to determine a new vergence depth for frame n+1 and to provide thenew vergence depth to varifocal actuation block 106. Based on the newvergence depth, varifocal element 802 moves optics block 104 fromposition p_(i) to new position p_(f) to accommodate user 800 at the newvergence depth d_(f) for the closer object 808.

In one example, each state of optics block 104 corresponds to acombination of focal length and eye position, provides accommodation fora range of vergence depths, and is associated with a specific positionof optics block 104. Accordingly, vergence depths may be mapped topositions of optics block 104 and stored in a lookup table. Thus, when avergence depth is received from vergence processing module 112,varifocal actuation block 106 automatically moves optics block 104 to aposition corresponding to the received vergence depth based on thelookup table.

In many instances, virtual reality systems aim to present users with avirtual environment that closely simulates a real world environment orprovides users with content causing the users to get lost in theillusion created by the virtual reality systems. To provide users with arealistic or captivating virtual environment, a virtual reality systemimplements multiple systems and methods discussed herein to operatetogether at efficiencies that are imperceptible to a user. For example,transition delays are particularly costly to user experience withvirtual reality systems. If a user is waiting for the virtual scenepresented by a VR headset to catch up to what the user's brain isalready expecting, the illusion is broken and/or the user may getnauseous. However, processing speeds and commercially availableactuators are currently faster than the coordination of the human eye tochange the shape of its lens and the human brain to register what thenew shape of the lens is focused on, allowing the disclosed systems andmethods to provide users with high-quality virtual environments.

Referring back to FIGS. 8A and 8B to provide accommodation for a newvergence depth while also leaving time to perform additionalcalculations without users perceiving a delay, a speed at whichvarifocal element 802 moves optics block 104 is limited by a rate atwhich the human eye performs accommodation. For example, assuming humaneye accommodation has a 10 diopter/sec peak velocity, a 100 diopter/sec²peak acceleration, and changing the distance between electronic display102 and optics block 104 moves a virtual image about 0.5 diopters mm,varifocal element 802 operates with a minimum velocity of 10/0.5=20mm/sec and a minimum acceleration of 100/0.5=200 mm/sec^(t) accelerationto prevent a user from perceiving the repositioning of optics block 104relative to electronic display 102. There are commercially availableactuators satisfying the preceding values.

FIGS. 9A and 9B show an alternative example process for adjusting thefocal length of optics block 104 with a varifocal element by changingthe shape of one or more lenses in optics block 104. Similarly to theexample of FIGS. 8A and 8B, FIG. 9A shows an example of VR headset 100providing focus for a frame n of a virtual scene that includes object902 displayed on electronic display 102 at which the gaze of user 900 isdirected (e.g., verged). A virtual image of object 902 is similarlylocated a virtual distance d₁, which is behind electronic display 102,from exit pupil 810. In the example of FIG. 9A, varifocal actuationblock 106 changes the shape of one or more lenses of optics block 104 toprovide accommodation for distance d_(i) with lens shape S₁ to enablecomfortable viewing of object 902.

FIG. 9B shows VR headset 100 providing focus for a subsequent frame n+1of the virtual scene, by changing the shape of one or more lenses ofoptics block 104. In this example, object 902 has moved toward user 900from distance d₁ to d₂ in the virtual scene, causing a change invergence and necessitating a complementary adjustment in accommodation.Accordingly, vergence processing module 106 determines a new vergencedepth for frame n+1 and provides the new vergence depth to varifocalactuation block 106, which changes the shape of one or more lenses ofoptics block 104 from shape S₁ to new lens shape S₂ to accommodate user900 at the new closer vergence depth d₂.

As described above, different states of optics block 104 correspond tovarious focal lengths, provide accommodation for a range of vergencedepths, and are associated with a lens shape or other adjustableproperty affecting focal length. Accordingly, vergence depths can bemapped to lens shapes or properties and stored in a lookup table. Thus,when a vergence depth is received from vergence processing module 112,varifocal actuation block 106 identifies a lens shape corresponding tothe vergence depth from the lookup table and changes the shape of one ormore lenses in optics block 104 to the identified lens shapecorresponding to the new vergence depth. As described above inconjunction with FIG. 1, varifocal actuation block 106 may include oneor more components to change the shape or other property affecting thefocal length of one or more lenses of optics block 104. Examplecomponents for changing the shape of one or more lenses or otherproperty affecting the focal length include: shape-changing polymerlenses, liquid lenses with electrowetting, Alvarez-Lohmann lenses,deformable membrane mirrors, liquid crystal (electroactive) lenses,phase-only spatial light modulator (SLM), and other suitable components.

Returning to FIG. 4, depth of field blur is determined 420 for thevirtual scene. To determine 420 depth of field blur, a point within thescene presented to the user by the VR headset 100 where the user's gazeis directed is determined, and optics block 104 is configured to a statein which the point within the scene where the user's gaze is directed isbrought into focus for the user. Depth of field blur is then determined420 relative to the point within the scene where the user's gaze isdirected. In one example, the depth within the scene geometry (e.g.,distances within the virtual scene) corresponding to the filteredvergence depth is determined as the plane of focus for the frame of thevirtual scene. Accordingly, objects or features of the virtualenvironment with distances within the virtual scene greater or less thana distance of the plane of focus from the user's eyes may be renderedwith synthetic blur. In another example, the depth of field blur isdetermined based on an object in the scene on which the user's gaze isfocused at the depth corresponding to the filtered vergence depth (i.e.,a “focal object”). Thus, the focal object, rather than the plane offocus, provides a reference point to identify other objects in the scenethat are rendered with depth of field blur, even if the other objectshave a similar depth in the scene as the focal object.

The blur may be progressive, with as a level of blur applied to objectsor features based on a distance of the objects or features from theplane of focus (or object of focus), or a generally uniform level ofblur may be applied to objects or features in the virtual scene. Depthof field blur is a natural consequence of binocular vision, so includingdepth of field blur in the virtual scene furthers to the illusion of thevirtual scene by providing the user with an expected depth cue, whichmay enhance the user experience with the virtual scene. Further, theblur may be based at least in part on measured properties of the user'seye. For example, wavefront aberrations of the user's eye could bemeasured by a wavefront aberrometer, with depth of field blur based atleast in part on the measured wavefront aberrations. Example wavefrontaberrations of the user's eye may include higher-order aberrations nottypically corrected by eye glasses, contact lenses, or refractivesurgery. Accounting for properties of the user's eye when determiningthe depth of field blur may improve user comfort when viewing the scene.

The frame of the virtual scene corresponding to the portion of thevirtual scene being viewed by the user is displayed 422 on electronicdisplay 102 with a distortion correction to correct optical error causedby the determined state of optics block 104 and with depth of field blurbased on the filtered vergence depth. Further, varifocal actuation block106 has changed the focus of optics block 104 to provide focus andaccommodation to the location in the portion of the virtual scene wherethe user's eyes are verged. In various embodiments, the process 400 mayinclude additional steps or perform steps in different orders than theorder described in conjunction with FIG. 4.

In various embodiments, operation of varifocal actuation block 106 isbased at least in part on characteristics of a user's vision. Forexample, optics block 104, varifocal actuation block 106 and scenerendering module 120 compensate for one or more characteristics of theuser's vision when a scene is presented to the user. Examplecharacteristics of a user's vision that may be compensated or otherwiseaccounted for by varifocal actuation block 106 include refractive errors(e.g., eyeglass prescription) and accommodation range (e.g., presbyopia,myopia, hyperopia or astigmatism). For example, if a user has arefractive error of +1 D spherical power, then optics block 104 isconfigured to a state that corrects the user's refractive error andmaintain focus, as well as maintains focus of objects within the sceneat which the user is looking. Additionally, varifocal actuation block106 and scene rendering module 120 may account for the user's specificaccommodation range (near to far distances with correction) whendetermining 420 depth of field blur. For example, users with presbyopiahave a reduced range of accommodation, so varifocal actuation block 106may provide a limited number of states of optics block 104 that accountfor a limited number of focal lengths or otherwise account for a user'srange of accommodation. Accommodation range and refractive errors may bespecified by the user or may be obtained from information associatedwith the user, such as digital medical records of the user that the userhas authorized one or more components of the VR system environment toaccess. Alternatively, lens assemblies or other components may becoupled to VR headset 100 to correct user's vision while using the VRheadset 100 instead of eyeglasses. If lens assemblies are coupled to VRheadset, 100, the lens assemblies may be separately calibrated to adjustthe correlation between states of optics block 104 and variousdistortion corrections. Further, varifocal actuation block 106 mayadjust the cylindrical power and axis of VR headset 100 to compensatefor astigmatism as well as adjust a spherical power of VR headset 100.For example, varifocal actuation block 106 rotates rotating twocylindrical lenses relative to one another to adjust the cylindricalpower of VR headset 100.

Display of a scene by VR headset 100 is modified to mitigate distortionintroduced by optical errors of optics block 104 included in VR headset100 that directs image light from electronic display element 102presenting the scene to an eye of a user. A distortion correction isapplied to the scene that pre-distorts the scene, and distortion causedby optics block 140 compensates for the pre-distortion as light from themodified scene passes through optics block 104. Hence, the scene viewedby the user is not distorted. Accordingly, distortion correctionsaccount for different levels and types of distortion caused by differenteye positions relative to optics block 104 or different focal lengths ofoptics block 140. Accordingly, the distortion corresponding to differentpotential eye positions relative to optics block 104 and at potentialfocal lengths of optics block 104 is determined by measuring a wavefront(i.e., propagation of points of the same phase) of light from theelectronic display element after the light has passed through the opticsblock Different eye positions relative to optics block 104 and differentstates of optics block 104 cause different degrees of optical error inlight directed through optics block 104. This optical error distortslight from electronic display element 102 included in VR headset 100,which may impair presentation of a virtual scene to a user. Accordingly,distortion correction maps are generated based on measurements of thewavefront for different states of optics block 104 to correct foroptical error introduced by different states of optics block 104, whichaccounts for different focal lengths caused by optics block 104.

Additional Configuration Information

The foregoing description of the embodiments has been presented for thepurpose of illustration; it is not intended to be exhaustive or to limitthe patent rights to the precise forms disclosed. Persons skilled in therelevant art can appreciate that many modifications and variations arepossible in light of the above disclosure.

The language used in the specification has been principally selected forreadability and instructional purposes, and it may not have beenselected to delineate or circumscribe the inventive subject matter. Itis therefore intended that the scope of the patent rights be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thepatent rights.

What is claimed is:
 1. A method comprising: determining, using at leastone image capturing element inside a virtual reality (VR) headset, aneye position for each eye of a user, the VR headset including an opticsblock configured to focus light from a display screen presenting avirtual scene to the user and a varifocal element configured to change afocal length of the optics block based on a viewing location within thevirtual scene of the user; determining gaze lines for each eye of theuser based at least in part on each eye position; determining a vergencedepth for the user based on an estimated intersection of the gaze lines;comparing the vergence depth to scene geometry data identifyingdistances within the virtual scene to generate a filtered vergencedepth; adjusting the focal length of the optics block based on thefiltered vergence depth to provide accommodation for the user using thevarifocal element; and displaying the virtual scene on the displayscreen of the VR headset.
 2. The method of claim 1, wherein displayingthe virtual scene on the display screen of the VR headset comprises:determining, for a frame of the virtual scene, a state of the opticsblock based on a state of the optics block during presentation of one ormore previously presented frames of the virtual scene, each state of theoptics block associated with a distortion correction map that correctsfor optical error introduced by the state of the optics block;determining a distortion correction map associated with the determinedstate of the optics block for the frame of the virtual scene to correctoptical error introduced by the determined state of the optics block forthe frame; applying the determined distortion correction map to theframe of the virtual scene; and displaying the virtual scene afterapplication of the determined distortion correction map on the displayscreen of the VR headset.
 3. The method of claim 2, further comprising:determining, for the determined eye position of each eye, at least oneof a distance between each eye and the optics block or a position ofeach eye relative to the optics block, each state of the optics block isassociated with a focal length of the optics block and at least one ofthe distance between each eye and the optics block or the position ofeach eye relative to the optics block, each distortion correction mapfurther correcting for optical error introduced by at least one of thedistance between each eye and the optics block or the position of eacheye relative to the optics block.
 4. The method of claim 1, whereindisplaying the identified objects in the virtual scene with syntheticblur comprises: identifying objects in the virtual scene with distancesfrom an eye of the user that are different than a distance of thefiltered vergence depth from the eye of the user; displaying theidentified objects in the virtual scene with synthetic blur determininga level of synthetic blur for an identified object based at least inpart on a distance between the identified object and the filteredvergence depth; and displaying the identified object in the virtualscene with the determined level of synthetic blur.
 5. The method ofclaim 1, wherein adjusting the focal length of the optics block based onthe filtered vergence depth to provide accommodation for the user usingthe varifocal element comprises: changing a distance between the opticsblock and the display screen by moving the optics block.
 6. The methodof claim 1, wherein adjusting the focal length of the optics block basedon the filtered vergence depth to provide accommodation for the userusing the varifocal element comprises: changing a shape or optical pathlength of a lens of the optics block.
 7. The method of claim 6, whereinchanging the shape or the optical path length of the lens of the opticsblock includes using at least one selected from a group consisting of ashape-changing polymer lens, a liquid lens and electrowetting, anAlvarez-Lohmann lens, a deformable membrane mirror, a liquid crystal(electroactive) lens, a phase-only spatial light modulator (SLM), andany combination thereof.
 8. The method of claim 1, wherein determining,using at least one image capturing element inside a virtual reality (VR)headset, an eye position for each eye of the user comprises: determininga position and an orientation of the VR headset using one or moreposition sensors; and determining the viewing location of the user basedat least in part on the position and the orientation of the VR headset.9. A method comprising: determining, using at least one image capturingelement inside a virtual reality (VR) headset, an eye position for eacheye of a user, the VR headset including an optics block configured tofocus light from a display screen presenting a virtual scene to an exitpupil and a varifocal element configured to change a focal length of theoptics block based on a viewing location within the virtual scene of theuser; determining a three-dimensional (3D) gaze point of the user basedat least in part on the eye position determined for each eye;determining a vergence depth for the user based on 3D gaze point, thevergence depth corresponding to an object of focus for the user;adjusting the focal length of the optics block for the virtual scenebased on the vergence depth using the varifocal element to provideaccommodation for the object of focus; and displaying, the virtual sceneon the display screen of the VR headset.
 10. The method of claim 9,wherein adjusting the focal length of the optics block for the virtualscene based on the vergence depth comprises: generating a filteredvergence depth by comparing the vergence depth to scene geometry dataidentifying distances within the virtual scene; and adjusting the focallength of the optics block for the virtual scene based at least in parton the filtered vergence depth.
 11. The method of claim 9, whereinadjusting the focal length of the optics block for the virtual scenebased on the vergence depth using the varifocal element comprises:changing a distance between the optics block and the display screen bymoving the optics block.
 12. The method of claim 9, wherein adjustingthe focal length of the optics block for the virtual scene based on thevergence depth using the varifocal element comprises: changing a shapeor optical path length of a lens included in the optics block.
 13. Themethod of claim 12, wherein changing the shape or optical path length ofthe lens included in the optics block comprises using at least oneselected from a group consisting of a shape-changing polymer lens, aliquid lens and electrowetting, an Alvarez-Lohmann lens, a deformablemembrane mirror, a liquid crystal (electroactive) lens, a phase-onlyspatial light modulator (SLM), and any combination thereof.
 14. Themethod of claim 9, wherein displaying the virtual scene on the displayscreen of the VR headset comprises: identifying objects in the virtualscene with distances from an eye of the user other than a distance ofthe filtered vergence depth from the eye of the user; and displaying theidentified objects in the virtual scene with synthetic blur.
 15. Anon-transitory computer-readable storage medium storing instructionsthat, when executed by at least one processor, cause the processor to:capture, using a camera of a virtual reality (VR) headset, imageinformation corresponding to an eye position for each eye of a user, theVR headset including an optics block configured to focus light from adisplay screen presenting a virtual scene to an exit pupil and avarifocal element configured to change a focal length of the opticsblock based on a viewing location of the user within the virtual scene;determine gaze lines of the user based at least in part on each eyeposition; determine a vergence depth for the user based on an estimatedintersection of the gaze lines, the vergence depth corresponding to aplane of focus for the user; compare the vergence depth to scenegeometry data identifying distances within the virtual scene to generatea filtered vergence depth; adjust the focal length of the optics blockfor a frame of the virtual scene using the varifocal element based onthe filtered vergence depth to provide accommodation for the plane offocus; and display the virtual scene on the display screen of the VRheadset
 16. The non-transitory computer-readable storage medium of claim15, wherein display the virtual scene on the display screen of the VRheadset comprises: determine, for a frame of the virtual scene, a stateof the optics block based on a state of the optics during presentationof one or more previously presented frames of the virtual scene, eachstate of the optics block associated with a distortion correction mapthat corrects for optical error introduced by the state of the opticsblock; determine a distortion correction map associated with thedetermined state of the optics block for the frame of the virtual sceneto correct optical error introduced by the determined state of theoptics block for the frame; apply the determined distortion correctionmap to the frame of the virtual scene; and display the virtual sceneafter application of the determined distortion correction map on thedisplay screen of the VR headset.
 17. The non-transitorycomputer-readable storage medium of claim 15, wherein display thevirtual scene on the display screen of the VR headset comprises:identify objects in the virtual scene with distances from an eye of theuser other than a distance of the filtered vergence depth from the eyeof the user; and display the identified objects in the virtual scenewith synthetic blur.
 18. The non-transitory computer-readable storagemedium of claim 15, wherein adjust the focal length of the optics blockbased on the filtered vergence depth to provide accommodation for theuser using the varifocal element comprises: change a distance betweenthe optics block and the display screen by moving the optics block. 19.The non-transitory computer-readable storage medium of claim 15, whereinadjust the focal length of the optics block based on the filteredvergence depth to provide accommodation for the user using the varifocalelement comprises: change a shape or optical path length of a lens ofthe optics block.
 20. The non-transitory computer-readable storagemedium of claim 19, wherein change the shape or optical path length ofthe lens of the optics block includes using at least one selected from agroup consisting of a shape-changing polymer lens, a liquid lens andelectrowetting, an Alvarez-Lohmann lens, a deformable membrane mirror, aliquid crystal (electroactive) lens, a phase-only spatial lightmodulator (SLM), and any combination thereof.